Visual constraints modulate stereotypical predictability of agents during situated language comprehension

We investigated how constraints imposed by the concurrent visual context modulate the effects of prior gender and action cues as well as of stereotypical knowledge during situated language comprehension. Participants saw videos of female or male hands performing an action and then inspected a display showing the faces of two potential agents (one male and one female face) as they listened to German OVS sentences about stereotypically female or male actions. Unlike previous experiments (Rodriguez et al., 2015), the display concurrent with the sentence also showed a picture of the object of the videotaped action and a ‘competitor object’ (with opposite stereotypical valence) that had not appeared in the video but could be mentioned in the sentence. We measured eye movements to the faces of the agents during comprehension. The design manipulated the match between the videotaped action and the action described by the sentence (actionverb(phrase) match) and the match between the stereotypical valence of the verbally described action and the gender of the agent of the previous video (conveyed only by the hands; stereotypicality match). We replicated the results obtained in Rodríguez et al. (2015): an overall target agent preference (i.e. the agent whose gender matched that of the hands seen in the previous video), reduced by action-verb mismatches. However, unlike in their study, mismatch effects emerged earlier. In addition, stereotypicality effects emerged in the verb region. The earlier mismatch effects and added stereotypicality effects suggest that the visual availability of the objects, perhaps jointly with the verbal input, facilitated the activation of representations from the recent videos (speeding up mismatch effects) and the consideration of alternative representations, favoring stereotypical expectations.